Offre en lien avec l’Action/le Réseau : MACLEAN/– — –

Laboratoire/Entreprise : Cnam/UBS
Durée : 6 months
Contact : nicolas.audebert@cnam.fr
Date limite de publication : 2023-02-01

Contexte :
Location : Cnam, Paris or UBS, Vannes (France)
Salary : ≈ 600€/month
Contract : Internship (4 to 6 months)
Starting date : Flexible in 2023

Research topic

Earth Observation through satellite imagery is a major tool for geoscience. However, there is a dilemma on whether to deploy systems that acquire many images at high frequency but with a low spatial resolution or systems with high spatial resolution but few revisits. The Sentinel-2 constellation, operated by the European Space Agency (ESA), leans towards high frequency for better temporal monitoring of dynamic phenomena. Yet, its 10m/px resolution is often not enough in urban applications since buildings, roads, and sparse vegetation are barely visible in the images. Conversely, commercial satellites, such as SPOT-6/7, often embark very high-resolution sensors. For example, the French mapping agency (IGN) distributes a yearly high-resolution cloud-free mosaic of the entire French metropolitan area. These panchromatic SPOT-6/7 images at a 1.5m/px resolution are helpful for precise land cover and land use mapping but are produced only once a year.

This research work aims to bridge the gap between those two data sources by investigating super-resolution. Super-resolution is an image processing technique aiming to increase the resolution of an image. It does not use a reference high-resolution image, but instead relies on prior knowledge regarding the structure of the data. Using deep learning for super-resolution has been frequently investigated in the last years [1], starting with Convolutional Neural Networks(CNN) [3], then later using Generative Adversarial Networks (GAN) [5, 8]. Nowadays, diffusion models [9, 6], already state-of-the-art generative models for image synthesis, have established themselves as strong candidates for the next generation of super-resolution algorithms.

Some works have transposed those deep architectures to remote sensing [4, 10], with two main shortcomings. First, most focus on single-image resolution, i.e., transforming a single image into a more detailed one. But low-resolution remote sensing data draws its strength from the temporal information contained in the satellite image time series (SITS). Second, super-resolution systems are mostly trained on artificial pairs of (low resolution, high resolution) images where the LR image is a downsampled version of the HR reference. This means that training sets and benchmarks are mostly synthetic and do not actually inform us of the performance of super-resolution systems on actual data. To overcome these issues, new benchmark datasets have been proposed such as Sen2Venus [7] and WorldStrat [2]. The latter includes pairs of a sequence of low-resolution Sentinel-2 images and a high-resolution Spot-6/7 image acquired worldwide. This dataset opens opportunities for the development of new approaches relying on modern state-of-the-art super-resolution techniques such as diffusion models.

Sujet :
The goal of this research project is (i) to leverage the temporal information contained in satellite time series to improve the super-resolution process, and (ii) to train deep diffusion models to power a super-resolution system on real data.

Objectives
The research intern will perform the following tasks:
• searching and summarizing the literature for suitable multi-image super-resolution approaches, diffusion models for super-resolution, and super-resolution of temporal data,
• collecting and analysing the WordStrat dataset,
• implementing and adapting state-of-the-art approaches, such as SRDiff [6], to SITS,
• investigating techniques to exploit the temporal structure in SITS, for example by using attention mechanisms and temporal blocks in the conditional encoder of diffusion models.

Profil du candidat :
The ideal applicant is pursuing an MSc. in Computer Science, Artificial Intelligence or Computer Vision or an equivalent degree (e.g. engineering diploma).

Formation et compétences requises :
• A good grasp of the fundamentals of machine learning and deep learning for computer vision is expected.
• The candidate should have an interest in scientific research and good written and oral communication skills.
• Knowledge of the Python programming language is a must, including some level of experience with at least one deep learning framework (PyTorch, Keras, TensorFlow, JAX…).
• A first experience with time series, image processing or generative models is a plus.
All applications, independently of previous experience, will be considered, provided that the candidate’s motivation and profile fit the internship topic. 
French is not required but can help with everyday life.

Adresse d’emploi :
This position is for an internship from 4 to 6 months, with a flexible starting date in 2023. The internship will either take place at the Center for research and studies in computer science and communications (Cédric), the computer science laboratory of the Conservatoire national des arts et métiers (Cnam) in Paris or at the Institute for Research in Informatics and Random Systems (Irisa), the computer science laboratory of the University of South Brittany in Vannes.
Depending on their preference, the intern will join the Complex Data, Machine Learning and Representations team in Paris or the Environment observation with complex imagery team in Vannes. In either cases, gratification is approximately 600€/month as imposed by the law for public institutions.